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ABSTRACT 

A discussion of testing methods in advanced English 
for Business and Technology looks at the differences between testing 
for this curriculum and testing in general English as a Second 
Language, drawing on testing theory and research. Two class 
assignments from the course are presented as examples of an 
assessment method used by the instructor. They include an initial 
writing assignment, started under supervision in a computer lab and 
completed independently, and a controlled computer lab assignment. 
The two assignments, each a memo, also include a response. It is 
proposed that testing at this level, in addition to meeting 
long-recognized test construction criteria, should be integrative: 
organized around a text and task, which entails a second, derivative 
text, and using pre-existing familiarity with the text and its 
factual and thematic background. Implications for classroom 
activities, curriculum design, and use of portfolios for student 
assessment are noted. Additional examples of assignments involving 
both individual and group work are presented. One student's response 
is used t'> illustrate both the effectiveness of the method and the 
types of testing and instructional issues that emerge from it. 
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In this paper I direct my attention to the role and mode of testing in advanced EBT 
(English for Business and Technology) courses which provide training in specialized written 
communication skills for prospective Business and Technology professionals. As I explore 
this issue and discuss my answer to it, I will in effect be addressing the issue raised by 3ohns 
and Dudley-Evans (1991), when they question if ESP (in contrast to general ESOL) 
methodology is possible. This is a more interesting question than might be evident at first 
glance. With regard to testing, I believe that I show that practice in ESP, specifically EST 
and EBT, can be methodologically principled. I also suggest that testing in EBT is 
appropriately different from modes of testing in general ESOL. At the same time, it is clear 
that general precepts of testing remain valid, and specific suggestions here are consonant 
with the standard framework as set forth, for example, by Brinton, Snow, and Wesche (p.3f, 
1989). 

Planning and implementing a syllabus for an advanced EBT course entails devising a 
series of detailed practical assignments each of which requires the student to construct 
practice workplace documents. The letters, reports, memoranda, etc. that students write in 
'•."'v response to course assignments often form the essential basis for assigning a course grade 



with little or no testing under academically secure conditions. This is not an unreasonable 
approach since the EBT course is supposed to be a writing practicum. Another factor that 



\ complicates testing in the EBT course is the fact that as the register being taught becomes 
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more specialized and student responses becomes more individualized, valid testing modes 
become harder to devise. 

In fact, testing should be integral to EBT courses and similar advanced ESP courses. 

In this paper I will discuss how testing can be integrated into the EBT course. I will maintain 
that devising testing strategies and the tests themselves should be a fundamental part of 
course design and management. 

Before proceeding with any discussion of testing theory or of the general, pedagogical 
motivation for the practical testing modes that I use, it would be useful to look at a concrete 
example of the sort of test that I have experimented with and found to be workable and valid. 
The following configuration consists of four "documents". These are 1) an initial writing 
assignment, started under supervision in a computer lab and completed independently; 2) a 
sample response; 3) a controlled computer lab writing assignment, effectively a test; 4) a 
sample response. 
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EXAMPLE 1. 



ESOL 3201/02 Assignment 3 Sept. 27, 1994 Elerick 

This assignment counts 6% of the course grade. 

The final draft of this assignment is due on Sept. 30. 

Write a memo to a superior (a Plant Manager or a Vice President, for example) in 
which you discuss some human factors problem that has come to light and for which you 
have primary responsibility for solving. 

As we have previously seen and discussed, the term "Human Factors " refers to the 
relation between the need for production and profitability and the need to maintain a healthy, 
productive, and cooperative w>.,.k force. Attention to human factors in the industrial and 
commercial workplace should simultaneously promote profitability and human satisfaction. 

Your memo might deal with any of the following. 

1. Safety: The avoidance of injury causing incidents 

2. Long-term physical well-being: Standing, moving, lifting; workers on 
assembly lines or repetitive motion problems for either standing or 
seated workers. 

3. Workplace conditions that may involve temperature, air quality, toxic 
chemicals, noise, etc. 

Solving human factor problems typically involves doing something that in the short 
run costs a company money. The assumption is always that this is money well spent, both for 
long term profitability and because it is the correct thing to do. 

In your memo suggest two things that you intend to do. One of these is to be a 
response that you make within a few days. The second is to be something that will take two 
to three weeks to plan and put in place. 

Students receive a copy of the following sample response at the time the assignment is 



given. 



Sept. 27, 1994 



TO: Robert Price, Plant Manager 
FROM: Heriberto Zufliga, Human Factors Engineer 
RE: Implementation of Safety Measures Detailed Below 

1 have reviewed the details surrounding the near-miss accident which occurred in the 
stamping shop on Sept. 21 and was discussed in the Supervisors' meeting on the 24th. 1 have 
determined that the following steps should be taken to avoid similar incidents in the future. 

1) All Maintenance and Production Foremen will be reminded that the power 
source lock-off requirement must be met before any piece of equipment can be 
worked on. 

2) This requirement will be extended and we will instruct all personnel to lock off 
the power source even before inspecting visually any apparent problem. This 
would have prevented the incident of the 21st. 

3) I am arranging a training and awareness session for Maintenance and 
Production Foremen so that the safety expert from our insurance carrier can 
talk to this group. 

We have not had a lost-time accident at the Moose Valley Plant in over four years. 1 
assure you that I am taking every measure to extend this record indefinitely. Please let me 
know if you have additional ideas to suggest. 



The students’ responses to this assignment were submitted on a Friday. The submitted 
work was corrected and selected examples were reviewed in class on the following Tuesday. 
On Thursday of the same week, members of the class, meeting in the writing lab, were 
handed the assignment and sample response that follow. The students knew that they would 
have a controlled directed writing exercise ("test") during that period and that it would, in 
some otherwise unspecified way, involve the computer file corresponding to the assignment 
from the previous week. 
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ESOL 3201/02 In-Class assignment Oct. 6, 1994 Elerick 

This assignment counts 6% of the course grade. 

The final draft of this exercise is due at the end of the class period. 

Write a follow-up memo to the same individual that you wrote to in your response to 
Assignment 3. Amend the course of action that was projected in your previous memo in the 
following ways. 

1) Report that some aspect of your near-time plan was not workable and could 
not be implemented. Relate what is to be done instead. 

OR 

Report some significant extension, proposed or implemented, of your 
previously described near-term course of action. 

THEN 

2) Report what you have done to implement that part of your plan that entailed 
some longer term arrangements. 



Oct. 4, 1994 

TO: Robert Price, Plant Manager 
FROM: Heriberto Zuhiga, Human Factors Department 
RE: Update on Safety Measures being taken or planned, ref. my memo of Sept. 27th. 

The power source lock-off procedures that 1 referred to in my earlier memo have been 
implemented. Frank Loya had an excellent idea and we have implemented it as well. For 
locking off the power source of any piece of machinery to be looked at or worked on, each 
repairman will have his own padlock. He will be personally responsible for locking off the 
power source and only he can remove the lock when he has finished his part of any 
inspection or repair. This should be fail-safe. 

The safety training and awareness session for Maintenance and Production Foremen 
could take place on Tuesday, Oct. 18th in the normal 11:00 - 12:00 AM meeting time. The 
Safety Consultant from Great Northern Casualty is available. Would that time be open as far 
as you are concerned? Do you think that any other supervisory personnel should attend such 
a session? 

Let me know what your thinking is regarding either of these matters. 
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The foregoing assignment/test specimen and the approach that it exemplifies should be 
considered in light of general precepts of academic testing and specifically those that inform 
practice in EBT. In general, tests that are given as part of organized courses should meet 
certain long-recognized criteria. For a general discussion of such criteria and of valid testing 
in ESOL see Kirschner, Wexler, and Spector-Cohen (1992). 

Al. The skills taught in the course and the type of materials used must be the basis of the 
examination. 

A2. The format of the test and the response should be familiar so that understanding the 
exam does not become the test. 

A3. Given Al and A2, the test should nevertheless be challenging. 

A4. The examination should constitute a learning experience in its own right. 

A5. The examination should, to every extent possible, deemphasize rote memory and 
emphasize instead analysis and process. 

A6. The test should rank students’ performance and should produce rankings that are not 
radically inconsistent with the general ranking of students’ performance in out-of-class 
writing assignments. Of course, if a student has been submitting work that does not 
reflect his or her own effort, the deviation may be significant. 

In addition to these general criteria for tests, examinations in high-intermediate and 
advanced ESOL classes should qualify as "integrative". For a good review of various 
practitioner’s thoughts regarding integrative testing see Weir (1990 p83ff). He notes that 
"Relatively little is known about integrative tests and even less is reported in the literature." 
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Citing the scant published material, Weir notes that at the lowest threshold integrative testing 
should involve multiple skills. But he quickly raises the expectation and suggests that a test 
which sustains a theme that reflects a realistic context is a superior instrument. He also 
notes ceitain shortcomings in integrative testing, including, among other factors, the 
difficulty of establishing the reliability of an integrative test and the difficulty of design and 
implementation. Both of these are important. Of course the question of validity is of 
significant concern when one needs to norm an instrument that tests general proficiency; 
questions of test validity can be dealt with much less formally when one uses an integrative 
examination in an intermediate or advanced academic course. 

The problem of implementation is another important issue that Weir raises. 
Appropriately, difficulty of implementation and how to respond to this issue is one of the 
main points of this paper. Implementation does dem.and close attention to details of 
assignment format and course planning. Planning that is anticipatory of integrative testing 
entails devising assignments that are consistent with the needs of those enrolled in the course 
and modeling these independently motivated assignments in ways that make them resources 
for test construction. In this way, feasibility and appropriateness in the construction of the 
syllabus coincide. 

In sum, the requirements for integrative testing are clear. To be integrative, an exam 
must be organized around a text and a task, which entails a second, related, derivative text. 
The student must work from pre-existing familiarity with the text and therefore the factual 
and thematic ground needed to deal with the task. With these advantages of prior familiarity, 
the student completing an integrative test is to be held to strict requirements regarding 
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intertextual assumption. 

A cursory review of the example of integrative ESP testing presented above reveals 
high compliance with both general and ESP-specific criteria. Given that this type of testing 
gets an initial "passing grade", we can proceed to a second set of considerations. We can 
now reflect on how this approach to testing serves not only the need to gather evaluative data 
in a secure mode, but also serves in some more integral way the purpose of the entire EBT 
course. Here are some ways in which this latter is accomplished. 

Bl. Course activities are more cohesive if weekly assignments and testing are related. 

Students who know that they may have to re-engage on any previous assignment are 
more likely to see the course as a whole than as a series of disparate 
assignments and exams. 

B2. EBT courses should simulate to the extent possible the conditions that students will ■ 
encounter after graduation in the workplace. As professionals, they will often have to 
re-engage on previous work and previously constructed documents, revising or 
extending these, using them as part of a larger document, or constructing a new 
document that presupposes the prior work and document. Testing by document 
re-engagement trains future professionals to do this. In this regard it might be noted 
that a controlled writing exercise should be completed in not much more time than a 
professional in the workplace would spend on a similar task. B3. The testing mode 
discussed here makes group work viable in that it provides for ways to test the 
individual team member’s grasp of the content, particularly the assumptions, of the 
group-prepared document. Group work is indispensable for the integrated 
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business/technology ESP class because it requires future product development and 
production personnel to engage with future management and marketing professionals. 
One academic drawback to group wo’’k is that all members of a team must be 
assigned the same grade. Reengagement testing can correct for this. The actual grade 
pairings for example III below was as follows. 

Group grade 75; Individual grades on test 65/80/75/75/90 
Group grade 95; Individual grades on test 65/95/90/95/90 
Group grade 90; Individual grades on test 80/80/90/80/70 
B4. There is much interest in the use, for various purposes, of student portfolios. 

Reengagement testing, with its resulting related student responses, provides excellent 
materials for the portfolio. 

With these testing desiderata and uses of integrative testing in mind, let us turn to an 



additional example. 



EXAMPLE II. 



ESOL 3201/02 Assignment 3 Feb. 12, 1994 Ch. Elerick 

Write a memo in which you confirm the agreement reached in a phone conversation 
that you have just had with the person to whom the memo is directed. 

The memo will deal with a malfunctioning piece of equipment that is in service in an 
office, laboratory, plant, or other work site environment where you have supervisory 
responsibility. The managerial superior to whom you will direct your memo has training and 
experience that is very similar to your own. 

Your memo is to include the following: 

1) An introductory sentence or two in which you reiterate the nature of the 
problem you are writing about. 

2) A precise identification of the equipment you are writing about. This should 
include technical information (model, etc.) and age and/or history of use. 
Other aspects of equipment history might be appropriate. 

3) A review and confirmation of the plan of action agreed to in the phone 
conversation. Here are some of the possibilities. 

A) The equipment can he repaired. You would specify what this would 
involve in time and expense and estimate the economic life of the 
equipment after the repairs. 

B) The equipment may have to be replaced. Reasons for replacing 
equipment normally involve one of the following. 

i) It cannot be (economically) repaired. 

ii) It can be repaired but not returned to a condition of long term 
serviceability. 

Hi) It is obsolete or obsolescent and a replacement built with state- 
of-the-art technology and features is indicated. 

4) Remind the person you are writing that if the course of action that you specify 
is not what he/she has understood to be the agreement, that he/she should 
contact you. 



The in-class assignment (test) took the following form. 
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ESOL 3201 In-class writing assignment April 28, 1994 



Review the memo that you wrote for Assignment 3. This will be on you course disk as 
Mn.S94FD. 

Write a second memo to the same person. Inform your supervisor that the solution to 
the equipment failure that you reported in your earlier memo cannot be implemeraed. 

Whether your solution involved repair or replacement, you will find suggestions for your 
follow-up memo below. Of course, you may devise your own reason. 

1) When the your maintenance personnel started to work on the problem, they 
discovered a new factor that made a simpler/more economical solution 
possible. 

2) When maintenance personnel started to work on the problem, they discovered 

a new and complicating factor that rendered the previous solution unworkable. 

a) More complete rebuilding is necessary. 

b) Repair is now seen as impractical and replacement of the equipment 
will be necessary. 

3) When repair work was begun, it became apparent that the repairs would take 
significantly longer than first anticipated. They will proceed but alternative 
plans for dealing with the equipment downtime will be needed. 

4) The. unavailability of repair parts or a delay in the delivery of parts makes a 
revision in the repair strategy impossible. 

5) Some complication in the. purchase of replacement equipment has arisen, 
necessitating an alternative. Complications could include delays in installation 
or an unsatisfactory service agreement that you were not aware of earlier. 

6) An alternative model that you have just been made aware of by a supplier’s 
representative motivates a late change of plans. 

When you write, a memo of this sort, you may have to say, in effect, that you made a 
mistake. Make yourself look good professionally by showing, indirectly, that the altered plan 
was necessitated by factors outside your control and that you have the self-confidence to 
identify a small and reasonable misjudgment before it becomes a larger mistake. 
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One additional example of integrative testing involves a group assignment that is 
followed up by an individual, in-class exercise. The initial assignment that supports the 
example adduced below assumes a consulting firm and involves a Report of some complexity 
that draws on the expertise of all the students on any given team. The follow up assignment, 
an in-class exercise, a test, serves to validate the participation of the students as individuals 
and tests how well each understands the work that has been done. At the time this test was 
given, the students did not know which prior assignment they would be required to reengage 
on; in this case, there had been an intervening assignment that had taken two weeks. 
EXAMPLE III 

ESOL 2301/02 Assignment 7 Dec. I, 1994 Elerick 
This assignment counts 12% of the final grade. 

Retrieve your file for the Group Report (GRnF94.FD) and review the work done at 
that time. Working as an individual, complete the following exercise. 

The firm that retained your consulting company has called and informed you that 
there is a problem regarding some aspect of the work that your firm did for them. For this 
assignment you will have to think of some problem that could have arisen, consistent with the 
prior problem/question and your firm’s solution. Go to the .site and investigate the situation. 

Write a short report in which you detail what you have determined to be the situation 
and either what you have done or what will need to be done. 

Deal with the problem with integrity. If your consulting group did make a mistake, do 
not try to conceal that fact. At the same time, write the report in a way that emphasizes the 
professionalism of the original work and of your follow-up. 

Cover your report with a letter of transmittal. 



A sample student response to this 80 minute exercise, with errors uncorrected, 
follows. 
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COMPU-SERVICES, INC. 



4500 Festival Ave. 
El Paso, TX 79923 
December 1, 1994 



Mr. Rene Williams 
Sales Representative 
Robinson Food Warehouse 
3848 Doniphan Avenue 
El Paso, TK 79901 



Dear Mr. Williams: 

The following report is in response of the problem with the new inventory-system at 
your company. 

I hope that our services and solutions to your problem meet your needs. I will be 
happy to answer any questions you might have regarding any of the procedures discussed in 
the report. 



Sincerely yours. 



Jaime Gdndara, President 
Compu-Services, Inc. 
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REPORT 



Yesterday morning, you requested a check-back of the new inventory system installed 
by us. According to the report you send us yesterday, the information printed by the server 
had a lot of errors. When you print the end of the day inventory movement you could only 
print out "garbage characters". Also, when displaying the information on the screen the same 
characters were seen. This type of problem was also detected by your personnel on the 
vjorkstation installed in the warehouse entrance. The only way you could print or display 
accurate inventory information was by using the portable TimeTech scanners. 

Yesterday, Roberto Torres our Systems Engineer and a group of technicians revised 
all the new inventory system. Since the problem was only with the hardware connected by 
coax cables we did an exhaustive testing of all communications devices. The test done to the 
software, (drivers), and hardware, (cards), of the communications devices showed no 
problems. The testing of the coax connecting cables showed many errors, so we proceed to 
revised the quality of them. As soon we knew the source of the problem we replace the coax 
cables by new ones. We did a new test to all communication devices once the new coax cable 
was installed. The printing and displaying of information by the end of the day was error- 
free. 



Our commitment as a consulting firm is to give the best and fastest service to our 
clients. The problem that you encounter could be avoided if we have checked the quality of 
the coax cables before installed them. For this 1 give my apologies and / hope that our fast 
solution to your problem was of your satisfaction. 
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Evaluating such student responses to controlled assignments that re-engage previous 
material is not significantly different from grading any similar assignment with the exception 
of one very important factor. The pre-existing material establishes a very clear set of pre- 
suppositions regarding the problem to be solved, what has been done to solve it, and the total 
configuration of players and factors. A response that does not correspond entirely to the 
presuppositions would be deemed deficient. The most important task of the instructor in the 
advanced, pre-professional writing course is to demand that students maintain consistency of 
pre-supposition, or manage pre-suppositional shifts, within a document and across related 
documents authored by the same individual or by correspondents. The testing mode 
suggested here puts a premium on the maintenance of pre-supposition and this should be 
prime in the evaluation of a test document. 

The sample document does show consistency with the pre-suppositional structure that 
supports the new document and it deserves a fairly high evaluation. The frequent verb errors 
constitute lower order errors and, though they detract, do not occasion a substantially 
lowered evaluation. The major problem in the response is the second sentence ["Since...] of 
the second paragraph, which is internally illogical and is not consistent with the argument 
structure of the paragraph. Other factors that detract from the grade are 1) the fact that the 
writer addresses the client at the beginning of the report and 2) the inclusion of material at 
the end of the report that should be expressed in the cover letter. 

The implementation of this approach to integrative testing and course planning 
requires the following: 
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Cl. The course must be a computer supported class which includes a supervised 
writing lab as a regular class activity. 

C2. Tests, represented as laboratory exercises, must be built into the course 
workplan. 

C3. An explicit student disk labeling and management must be in place at the 
start of the course. Each student must be assigned a number and then maintain 
a disk with file designations similar to that which follows. The file labeling 
plan must be established at the beginning of the course and closely monitored. 



Assignment 1 
2 

3 

4 

5 

6 

7 

8 
9 



PnS95.D 

MnS95.D 

HFnS95.D 

RnS95.D 

TnS95 

GRnS95.D 

SnS95.D 

TXnS95 

FnS95 



PnS95.FD 

MnS9.5.FD 

HFnS95.FD 

RnS95.FD 

GRnS95.FD 

SnS95.FD 



This testing mode can be extended to any pairs of documents that meet the general 
requirements reviewed here. There must be an earlier assignment with a resulting document 
that can b-e complemented by a second closely related and derivative document. This second 
document must be of a sort that a student can prepare it within a single class period and with 
no access to research sources or data. Three additional epitomized examples of such pairs of 
documents follow. 
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Assignment: 

Test: 

Assignment: 

Test: 

Assignment: 

Test: 



Prepare a Preliminary Proposal (typically a group assignment). 

Write a 5(X) word Executive Summary for the Proposal. 

Write an Interim Report (60 day) for a project that is expected to take 
180 days. 

Write a 120 day report for the same project. 



Write a short report for your plant manager in which you propose and 
justify one minor but significant revision in some aspect of company 
policy affecting employees. 

Assuming that the proposed policy change has been approved, write a 
memo to the affected employees that states what the new policy is and 
how it will be implemented. 
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